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ABSTRACT 

The  U.S.  military  has  been  conducting  psychological 
screening  of  deploying  and  redeploying  troops  for  a 
number  of  years  for  early  identification  of  service 
members  at  risk  for  mental  health  problems.  Currently 
research  is  being  conducted  to  assess  the  validity  of  the 
clinical  domains  included  on  the  screening  instrument.  As 
part  of  this  systematic  assessment,  a  blind  validation  study 
was  conducted  in  March  2004  with  soldiers  returning  from 
a  year  long  deployment  to  Iraq.  The  results  of  this  study 
are  summarized  for  each  clinical  domain,  detailing  the 
development  of  a  short,  but  valid  and  comprehensive 
screening  instrument. 

1.  INTRODUCTION 

Mental  health  problems  are  some  of  the  most  common 
and  disabling  medical  conditions  that  affect  service 
members.  Among  active  duty  U.S.  military  service 
members,  mental  disorders  are  the  leading  cause  of 
hospitalization  for  men  and  the  second  leading  cause  for 
women  (second  only  to  pregnancy-related  admissions). 
Six  to  ten  percent  of  U.S.  military  personnel  receive 
outpatient  treatment  for  a  mental  disorder  each  year  (Hoge 
et  al.,  2002;  Garvey-Wilson  et  ah,  2003).  Over  25%  of 
service  members  who  receive  outpatient  care  for  mental 
health  problems  leave  military  service  within  six  months,  a 
rate  that  is  more  than  two  times  higher  than  the  rate  of 
attrition  following  treatment  for  any  other  ICD-9  illness 
category  (Hoge  et  ah,  2002). 

Psychiatric  health  is  also  a  concern  in  operational  and 
post-operational  environments.  During  Operation 
Enduring  Freedom  (OEF)  and  Operation  Iraqi  Freedom 
(OIF),  approximately  7%  of  evacuations  from  the  theater 
were  listed  as  having  a  primary  psychiatric  diagnosis. 
Many  studies  have  demonstrated  the  strong  link  between 


deployment  experiences,  especially  combat,  and  a  variety 
of  adverse  mental  health,  psychosocial,  and  occupational 
effects,  including  PTSD  (15-40%  lifetime  rate  after 
combat),  depression,  substance  abuse,  job  loss, 
unemployment,  divorce,  and  severe  spouse  abuse  (Centers 
for  Disease  Control,  1998;  Iowa  Persian  Gulf  Study 
Group,  1997;  Prigerson,  Maciejewski  &  Rosenheck,  2002; 
McCarroll,  Ursano,  Liu,  Thayer,  Newby,  Norwood  & 
Fullerton,  2000).  Available  data  also  indicate  that  most 
service  members  with  mental  health  concerns  do  not  seek 
treatment,  due  to  stigma  and  other  barriers,  although  very 
limited  research  has  been  conducted  in  this  area  (Bray, 
Sanchez,  Ornstein  et.  ah  1998;  Hoge,  2003). 

Given  the  obvious  importance  of  mental  disorders 
among  military  service  members  and  the  unusual  stressors 
experienced  during  deployment,  it  would  be  desirable  to 
have  a  simple  and  cost  effective  ways  to  identify  Soldiers 
at  risk  of  behavioral  health  problems.  Toward  this  goal, 
the  U.S.  military  has  been  conducting  psychological 
screening  of  deploying  and  returning  troops  for  a  number 
of  years.  In  1996,  the  U.S.  military  mandated  that 
psychological  screening  be  conducted  before  and  after 
deployment  to  the  Balkans  in  order  to  facilitate  early 
identification  of  mental  health  problems  in  redeploying 
soldiers.  Although  the  mandate  was  lifted  in  1999,  many 
units  participating  in  deployment  operations  opted  to 
maintain  psychological  screening  as  a  part  of  their  before 
and  after-care  procedures.  Recently,  units  deploying  to 
and  returning  from  Afghanistan  and  Iraq  have  also 
included  psychological  screening  as  part  of  their  post¬ 
deployment  mental  health  assessment  in  the  form  of  the 
mandated  DD  Form  2796. 

Despite  the  longevity  of  the  psychological  screening 
program,  relatively  little  research  has  been  conducted  to 
determine  the  validity  of  the  specific  psychological 
screening  instalments  used  to  identify  symptomatic 
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Soldiers.  Historically,  screening  instruments  have  tended 
to  be  selected  based  on  validity  work  conducted  in  civilian 
populations.  For  instance,  one  might  screen  for  depression 
using  an  instrument  developed  and  widely  used  in  civilian 
settings.  When  these  instruments  have  been  applied  to 
Soldiers,  the  scoring  procedures  developed  in  civilian 
populations  are  likewise  applied  to  the  Soldier  populations. 

It  is  certainly  logical  to  develop  screening  instruments 
using  established  instruments  from  civilian  research; 
nonetheless,  independent  assessments  of  the  properties  of 
these  instillments  using  military  populations  are  important 
for  two  reasons.  First,  it  is  possible  that  scoring  criteria 
established  in  civilian  populations  might  not  be  directly 
applicable  to  military  settings,  so  validity  work  with 
military  populations  can  address  the  generalizability  of 
scoring  criteria.  The  second  reason  why  it  is  important  to 
further  assess  these  instruments  is  because  scales 
developed  in  civilian  populations  and  used  in  civilian 
settings  are  often  lengthy  and  adopt  complex  scoring 
algorithms.  In  civilian  settings,  neither  length  nor 
complexity  are  particularly  problematic,  because  the 
instillments  are  rarely  administered  to  more  than  two  or 
three  individuals.  In  addition,  in  many  civilian  settings  the 
administration  and  scoring  can  easily  be  computerized 
reducing  concerns  about  complex  scoring  algorithms. 

In  the  military,  however,  both  length  and  complexity 
are  more  problematic.  Instillments  with  15-25  items  per 
dimension  can  add  up  to  lengthy  primary  screening 
surveys  when  multiple  dimensions  (depression,  traumatic 
stress,  alcohol,  etc.)  are  assessed.  In  addition,  in  many 
screening  settings,  individuals  (not  computers) 
immediately  score  the  primary  screen,  and  given  the  size 
of  the  military  these  individuals  may  score  thousands  of 
surveys  in  a  matter  of  days.  Consequently,  length  and 
scoring  complexity  are  more  problematic  because  they  add 
time  and  error  into  the  screening  process.  Consequently, 
one  goal  of  screening  research  is  to  build  upon  existing 
screening  instruments  and  establish  the  validity  of  short  (3- 
5  items)  and  easily  scored  scales. 

2.  PROCEDURE 

To  specifically  address  the  issues  of  validity, 
instillment  length  and  scale  complexity,  we  conducted  a 
blind-validation  study  within  a  screening  assessment 
recommended  by  a  senior  military  operational  commander. 
The  blind  validation  screening  study  was  conducted  in 
March  2004  with  1,578  Soldiers  returning  from  a  year¬ 
long  deployment  to  Iraq.  In  the  study,  Soldiers  completed 
a  primary  screen  survey  assessing  dimensions  of  (a) 
Traumatic  Stress,  (b)  Depression,  (c)  Anger,  (d) 
Relationship  problems  and  (e)  Alcohol  problems.  Several 
of  the  dimensions  contained  items  from  two  or  more 
scales.  For  instance,  the  traumatic  stress  dimension 


contained  items  from  the  DD  Form  2796  (the  DOD  Post- 
Deployment  Health  Assessment),  as  well  as  17  items  from 
the  PTSD  Checklist  (PCL)  developed  by  the  National 
Center  for  PTSD  (Weathers,  et  ah,  1993). 

Using  previously  established  criteria.  Soldiers  who 
scored  positively  on  any  one  of  the  five  clinical 
dimensions  (a-e)  were  instructed  to  undergo  a  secondary 
screening  interview  with  a  clinical  provider.  Using  a 
validated  structured  interview  format  (Sheehan  et  al. 
1998),  the  clinical  provider  determined  whether  or  not  the 
Soldier  required  a  mental  health  referral.  To  support  the 
validation  aspect  of  the  work,  however,  clinical  providers 
were  kept  blind  to  the  results  of  the  primary  screen  when 
conducting  secondary  interviews.  In  addition,  a  sample  of 
approximately  20%  of  the  Soldiers  scoring  negative  on  all 
domains  was  randomly  assigned  the  secondary  interview 
condition.  These  procedures  ensured  that  clinical 
providers  knew  nothing  about  the  status  of  the  Soldier 
prior  to  conducting  the  interview.  In  this  way,  the  primary 
screen  could  be  validated  by  determining  how  well  the 
items  and  scales  on  the  primary  screen  corresponded  with 
clinical  providers’  evaluations. 

3.  RESULTS 

The  analyses  are  based  upon  the  congruence  between 
(a)  Soldiers’  responses  on  the  primary  screen  and  (b) 
clinical  providers’  evaluations.  This  congruence  can  be 
summarized  in  terms  of  sensitivity  and  specificity. 
Sensitivity  represents  the  proportion  of  Soldiers  clinical 
providers  identify  with  symptoms  who  were  also  positive 
on  the  primary  screening  instrument.  Specificity 
represents  the  proportion  of  Soldiers  without  symptoms 
who  had  negative  test  results.  As  a  point  of  reference,  we 
generally  consider  a  test  to  be  acceptable  if  both  sensitivity 
and  specificity  are  .70  or  above. 

In  the  following  sections,  a  summary  of  results  is 
provided  for  each  clinical  dimension  assessed  in  the 
screening  process  except  alcohol  problems.  Alcohol 
problems  were  not  assessed  because  Soldiers  had  no 
access  to  alcohol  during  the  preceding  12  months,  and 
alcohol  questions  always  focus  on  prior  behavior  (drinking 
behavior  in  the  last  month  or  year). 

3.1  Traumatic  Stress 

Results  showed  that  the  four  PTSD-related  items  on 
question  12  of  the  DD  Form  2796  did  a  good  job  of 
identifying  symptomatic  Soldiers.  The  stem  for  the 
question  read  “Have  you  ever  had  any  experience  that  was 
so  frightening,  horrible,  or  upsetting  that,  IN  THE  PAST 
MONTH,  you...(l)  Had  nightmares  about  it  or  thought 
about  it  when  you  did  not  want  to?  (2)  Tried  hard  not  to 
think  about  it  or  went  out  of  your  way  to  avoid  situations 
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that  remind  you  of  it?  (3)  Were  constantly  on  guard, 
watchful,  or  easily  startled?  and  (4)  Felt  numb  or  detached 
from  others,  activities,  or  your  surroundings?  With  two 
positive  responses,  the  sensitivity  of  these  items  as  a 
screen  was  0.73,  and  the  specificity  was  0.88. 
Interestingly,  the  four  items  did  as  well  as  the  more 
extensive  17-item  scale  (Weathers,  et  al.,  1993)  using  cut¬ 
off  criteria  published  in  the  scientific  literature.  For 
complete  results  see  Bliese,  Wright,  Adler,  Thomas  & 
Hoge  (2004). 

3.2  Depression 

An  analyses  of  two  depression  scales  (Self-Rating 
Depression  Scale  or  SDS,  Zung,  1965;  and  the  Patient 
Health  Questionnaire  or  PHQ,  Spitzer  et  al.,  1999) 
revealed  that  four  PHQ  items  could  be  used  as  an  effective 
primary  screen.  The  stem  for  the  items  was:  “Over  the 
last  2  weeks,  how  often  have  you  been  bothered  by  any  of 
the  following  problems”,  (1)  Little  interest  or  pleasure  in 
doing  things,  (2)  Feeling  down,  depressed,  or  hopeless,  (3) 
Poor  appetite  or  overeating,  and  (4)  Trouble  concentrating 
on  things  such  as  reading  the  newspaper  or  watching 
television.  If  a  Soldier  reported  symptoms  more  than  half 
the  days  for  at  least  one  item,  the  sensitivity  was  0.77  and 
the  specificity  was  0.88.  Further  details  are  reported  in 
Bliese,  Wright,  Adler  &  Thomas  (2004). 

3.3  Anger 

Based  on  various  anger  items  from  the  literature  (e.g., 
Buss  &  Perry,  1992),  three  items  were  found  to  create  an 
effective  anger  screen.  The  stem  for  the  questions  was: 
“During  the  PAST  MONTH,  how  often  have  you  been 
bothered  by  any  of  the  following  problems”  (1)  Became 
so  angry  that  you  have  broken  things,  (2)  Was  on  the  verge 
of  losing  control  of  your  anger,  and  (3)  Flew  off  the  handle 
for  no  good  reason.  When  a  Soldier  reported  Sometimes, 
Often  or  Very  Often  to  one  or  more  of  the  items,  the 
sensitivity  of  the  test  was  0.74,  and  the  specificity  was 
0.86.  Further  details  are  reported  in  Bliese,  et  al.,  (2004). 

3.4  Relationship  Problems 

Two  items  were  found  to  be  an  acceptable  screen  for 
relationship  problems.  The  first  item  was  “Are  you  having 
marital  or  relationship  problems  (Y  or  N)?”  and  the  second 
item  was  “Our  relationship  is  strong”  (Strongly  Disagree, 
Disagree,  Neutral,  Agree,  and  Strongly  Agree).  A  positive 
response  on  the  first  item  and  Neutral,  Disagree,  or 
Strongly  Disagree  response  on  the  second  item 
corresponded  to  a  sensitivity  value  of  0.82  and  a 
specificity  value  of  0.89.  Further  details  are  reported  in 
Bliese,  et  al.,  (2004). 


4.  DISCUSSION 

By  building  a  blind  validation  procedure  into  a 
screening  program  requested  by  an  operational 
commander,  we  were  able  to  develop  four  very  short 
screens  and  validate  the  utility  of  an  instrument  in  use  (in 
the  case  of  question  12  on  the  DD  Form  2796  for  traumatic 
stress).  The  value  of  this  research  is  that  it  moves  us 
significantly  closer  to  our  goal  of  developing  a  short,  but 
valid  and  comprehensive  screening  tool  that  can  quickly  be 
taken  by  Soldiers  and  easily  scored  by  technicians.  Our 
goal  is  to  field  such  an  instrument  on  a  large  scale  within 
the  next  year. 

5.  CONCLUSION 

Mental  health  problems  have  been  identified  as  the 
most  important  medical  correlate  of  attrition  from  military 
service,  as  well  as  a  concern  in  operational  and  post- 
operational  environments.  Early  identification  and 
intervention  with  Soldiers  at  risk  for  developing  such 
problems  will  lead  to  referrals  to  the  appropriate  service  or 
program  and  may  reduce  the  severity  and  chronicity  of  the 
problem  and  the  corresponding  distress.  Currently  the  US 
Army  is  systematically  validating  a  psychological 
screening  instalment  that  can  be  linked  with  improved 
prevention  efforts.  The  recent  blind  validation  study 
conducted  with  Soldiers  returning  from  Iraq  assessed  the 
congruence  between  the  scales  on  the  screening  instrument 
and  clinical  providers’  evaluations.  This  study  resulted  in 
sensitivity  and  specificity  analyses  for  each  of  the  clinical 
domains  on  the  screening  instrument.  Four  shortened 
scales  with  acceptable  sensitivity  and  specificity  were 
developed  and  will  be  further  validated  and  fielded  over 
the  next  year. 
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